Often, various applications will display a visual representation that corresponds to a user that the user controls through certain actions, such as selecting buttons on a remote or moving a controller in a certain manner. The visual representation may be in the form of an avatar, a fanciful character, a cartoon image or animal, a cursor, a hand, or the like. The visual representation is a computer representation corresponding to a user that typically takes the form of a two-dimensional (2D) or three-dimensional (3D) model in various applications, such as computer games, video games, chats, forums, communities, instant messaging services, and the like. Many computing applications such as computer games, multimedia applications, office applications, or the like provide a selection of predefined animated characters that may be selected for use in the application as the user's avatar. Some systems may incorporate a camera that has the ability to take a picture of a user and identify features from that frame of data. However, these systems require a capture of a user's feature, processing of the image, and then application to the character in a non-real time environment, and the features applied are low fidelity, usually based on a single snapshot of the user.